Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation

Convolutional neural networks have been widely deployed in various application scenarios. In order to extend the applications' boundaries to some accuracy-crucial domains, researchers have been investigating approaches to boost accuracy through either deeper or wider network structures, which brings with them the exponential increment of the computational and storage cost, delaying the responding time. In this paper, we propose a general training framework named self distillation, which notably enhances the performance (accuracy) of convolutional neural networks through shrinking the size of the network rather than aggrandizing it. Different from traditional knowledge distillation - a knowledge transformation methodology among networks, which forces student neural networks to approximate the softmax layer outputs of pre-trained teacher neural networks, the proposed self distillation framework distills knowledge within network itself. The networks are firstly divided into several sections. Then the knowledge in the deeper portion of the networks is squeezed into the shallow ones. Experiments further prove the generalization of the proposed self distillation framework: enhancement of accuracy at average level is 2.65%, varying from 0.61% in ResNeXt as minimum to 4.07% in VGG19 as maximum. In addition, it can also provide flexibility of depth-wise scalable inference on resource-limited edge devices.Our codes will be released on github soon.Comment: 10page

Satellite-detected ammonia changes in the United States: Natural or anthropogenic impacts

Ammonia (NH3) is the most abundant alkaline component and can react with atmospheric acidic species to form aerosols that can lead to numerous environmental and health issues. Increasing atmospheric NH3 over agricultural regions in the US has been documented. However, spatiotemporal changes of NH3 concentrations over the entire US are still not thoroughly understood, and the factors that drive these changes remain unknown. Herein, we applied the Atmospheric Infrared Sounder (AIRS) monthly NH3 dataset to explore spatiotemporal changes in atmospheric NH3 and the empirical relationships with synthetic N fertilizer application, livestock manure production, and climate factors across the entire US at both regional and pixel levels from 2002 to 2016. We found that, in addition to the US Midwest, the Mid-South and Western regions also experienced striking increases in NH3 concentrations. NH3 released from livestock manure during warmer winters contributed to increased annual NH3 concentrations in the Western US. The influence of temperature on temporal evolution of NH3 concentrations was associated with synthetic N fertilizer use in the Northern Great Plains. With a strong positive impact of temperature on NH3 concentrations in the US Midwest, this region could possibly become an atmospheric NH3 hotspot in the context of future warming. Our study provides an essential scientific basis for US policy makers in developing mitigation strategies for agricultural NH3 emissions under future climate change scenarios

Effects of detection-beam focal offset on displacement detection in optical tweezers

A high-resolution displacement detection can be achieved by analyzing the scattered light of the trapping beams from the particle in optical tweezers. In some applications where trapping and displacement detection need to be separated, a detection beam can be introduced for independent displacement detection. However, the detection beam focus possibly deviates from the centre of the particle, which will affect the performance of the displacement detection. In this paper, we detect the radial displacement of the particle by utilizing the forward scattered light of the detection beam from the particle. The effects of the lateral and axial offsets between the detection beam focus and the particle centre on the displacement detection are analyzed by the simulation and experiment. The results show that the lateral offsets will decrease the detection sensitivity and linear range and aggravate the crosstalk between the x-direction signal and y-direction signal of QPD. The axial offsets also affect the detection sensitivity, an optimal axial offset can improve the sensitivity of the displacement detection substantially. In addition, the influence of system parameters, such as particle radius a, numerical aperture of the condenser NAc and numerical aperture of the objective NAo on the optimal axial offset are discussed. A combination of conventional optical tweezers instrument and a detection beam provides a more flexible working point, allowing for the active modulation of the sensitivity and linear range of the displacement detection. This work would be of great interest for improving the accuracy of the displacement and force detection performed by the optical tweezers.Comment: 10 pages,11 figure

Flexible planar metamaterials with tunable Poisson’s ratios

This research reports on the design, fabrication, and multiscale mechanical characterization of flexible, planar mechanical metamaterials with tailorable mechanical properties. The tunable mechanical behavior of the structures is realized through the introduction of orthogonal perforations with different geometric features. Various configurations of the perforations lead to a wide range of Poisson’s ratios (from −0.8 to 0.4), load-bearing properties, and energy absorption capacities. The correlations between the configuration of the perforations and the auxetic response of the structures are highlighted through computational and experimental characterizations performed at multiple length scales. It is demonstrated that the local in-plane rotation of the solid ligaments in a uniaxially loaded structure is the primary factor that contributes to its strain-dependent auxetic behavior at macroscopic scales. Confinement of these local rotations is then used as a practical strategy to activate a self-strengthening mechanism in the auxetic structures. It is further shown that the fabrication of planar flexible structures with controllable Poisson’s ratios is feasible through spatial adjustment of perforations in the structure. Finally, discussions are provided regarding the practical applications of these structures for a new generation of highly energy-absorbing protective equipment

Flexible planar metamaterials with tunable Poisson\u27s ratios

This research reports on the design, fabrication, and multiscale mechanical characterization of flexible, planar mechanical metamaterials with tailorable mechanical properties. The tunable mechanical behavior of the structures is realized through the introduction of orthogonal perforations with different geometric features. Various configurations of the perforations lead to a wide range of Poisson\u27s ratios (from −0.8 to 0.4), load-bearing properties, and energy absorption capacities. The correlations between the configuration of the perforations and the auxetic response of the structures are highlighted through computational and experimental characterizations performed at multiple length scales. It is demonstrated that the local in-plane rotation of the solid ligaments in a uniaxially loaded structure is the primary factor that contributes to its strain-dependent auxetic behavior at macroscopic scales. Confinement of these local rotations is then used as a practical strategy to activate a self-strengthening mechanism in the auxetic structures. It is further shown that the fabrication of planar flexible structures with controllable Poisson\u27s ratios is feasible through spatial adjustment of perforations in the structure. Finally, discussions are provided regarding the practical applications of these structures for a new generation of highly energy-absorbing protective equipment

Enhanced Interfacial Dzyaloshinskii-Moriya Interaction in annealed Pt/Co/MgO structures

The interfacial Dzyaloshinskii-Moriya interaction (iDMI) is attracting great interests for spintronics. An iDMI constant larger than 3 mJ/m^2 is expected to minimize the size of skyrmions and to optimize the DW dynamics. In this study, we experimentally demonstrate an enhanced iDMI in Pt/Co/X/MgO ultra-thin film structures with perpendicular magnetization. The iDMI constants were measured using a field-driven creep regime domain expansion method. The enhancement of iDMI with an atomically thin insertion of Ta and Mg is comprehensively understood with the help of ab-initio calculations. Thermal annealing has been used to crystallize the MgO thin layer for improving tunneling magneto-resistance (TMR), but interestingly it also provides a further increase of the iDMI constant. An increase of the iDMI constant up to 3.3 mJ/m^2 is shown, which could be promising for the scaling down of skyrmion electronics

Segmentation of kidney lesions with attention model based on Deeplab

We participate this challenge by developing a hierarchical framework. We build the model from two fully convolutional networks: (1) a simple Unet model to normalize the input iamges, (2) a segmentaion network which is an attention model based on Deeplab model. Two models are connected in tandem and trained end-to-end. To ensure a better results, we use the preprocess method proposed by nnUnet in our experiments

IIV-6 Inhibits NF-kappaB Responses in Drosophila

The host immune response and virus-encoded immune evasion proteins pose constant, mutual selective pressure on each other. Virally encoded immune evasion proteins also indicate which host pathways must be inhibited to allow for viral replication. Here, we show that IIV-6 is capable of inhibiting the two Drosophila NF-kappaB signaling pathways, Imd and Toll. Antimicrobial peptide (AMP) gene induction downstream of either pathway is suppressed when cells infected with IIV-6 are also stimulated with Toll or Imd ligands. We find that cleavage of both Imd and Relish, as well as Relish nuclear translocation, three key points in Imd signal transduction, occur in IIV-6 infected cells, indicating that the mechanism of viral inhibition is farther downstream, at the level of Relish promoter binding or transcriptional activation. Additionally, flies co-infected with both IIV-6 and the Gram-negative bacterium, Erwinia carotovora carotovora, succumb to infection more rapidly than flies singly infected with either the virus or the bacterium. These findings demonstrate how pre-existing infections can have a dramatic and negative effect on secondary infections, and establish a Drosophila model to study confection susceptibility

Signal identification with Kalman Filter towards background-free neutrinoless double beta decay searches in gaseous detectors

Particle tracks and differential energy loss measured in high pressure gaseous detectors can be exploited for event identification in neutrinoless double beta decay~($0\nu \beta \beta$) searches. We develop a new method based on Kalman Filter in a Bayesian formalism (KFB) to reconstruct meandering tracks of MeV-scale electrons. With simulation data, we compare the signal and background discrimination power of the KFB method assuming different detector granularities and energy resolutions. Typical background from $^{232}$Th and $^{238}$U decay chains can be suppressed by another order of magnitude than that in published literatures, approaching the background-free regime. For the proposed PandaX-III experiment, the $0\nu \beta \beta$ search half-life sensitivity at the 90\% confidence level would reach $2.7 \times 10^{26}$~yr with 5-year live time, a factor of 2.7 improvement over the initial design target

The Surface Processes on Ru/Pt(111) as Probed by Cyclic Voltammetry and in Situ Surface-Enhanced Raman Spectroscopy

Information about the chemical adsorption and surface oxidation of Ru under electrochemical conditions is of great importance for understanding the structure–activity relationship in Ru-based materials. Quasi-single-crystalline Ru films on single-crystal Pt(111) electrodes (Ru/Pt(111)) were prepared by the forced-deposition method along with inductive heating treatment. The adsorption of both hydrogen and oxygen species on Ru/Pt(111) was studied by cyclic voltammetry and CO displacement. The potential of zero total charge on Ru/Pt(111) is ca. 0.12 V. A detailed study on oxygen species was carried out by in situ surface-enhanced Raman spectrometry. Ru–O was found to form at E > 0.1 V, and the conversion of Ru–O into RuO2 occurred at E = 0.3 V. The reversible oxidation occurs up to 1.0 V. Our results suggest that Ru/Pt(111), which exhibits electrochemical properties similar to those of Ru(0001), may serve as an alternative for Ru study as well as a model system for understanding ligand and strain effects.We gratefully acknowledge the funding by the China Scholarship Council (CSC), the National Natural Science Foundation of China (nos. 22172151 and 21972131), the Ministerio de Ciencia e Innovación (Spain) grant nos. PID2019-105653GB-I00 and FJC2018-038607-I, and Generalitat Valenciana (Spain) grant number PROMETEO/2020/063